Heterogeneity of Treatment Effects in Internet- and Mobile-Based Interventions for Depression

Key Points Question Is there evidence from randomized clinical trials (RCTs) that patients respond differently to internet- and mobile-based interventions (IMIs) for depression? Findings In this meta-analysis of 102 RCTs involving 19 758 participants, clinically relevant effect sizes for unguided IMIs in patients with subthreshold to mild depression without evidence for substantial patient-by-treatment interaction was found. In contrast, heterogeneity of treatment effects and moderating effects of guidance increased with baseline depression severity. Meaning These findings suggest that moderate improvements in subthreshold to mild depression can be reasonably expected from unguided IMIs, but individuals with more severe depression could respond differently, indicating the need for digital precision psychotherapy and future research in this subgroup.

This supplemental material has been provided by the authors to give readers additional information about their work.

eAppendix 1. Illustration of Mean and Variance Differences in an RCT
Assuming a randomized controlled trial with one intervention group (IG) and one control group (CG), the mean (e.g., an average of depression severity) and standard deviation (e.g., the sum of the individual differences from the mean) are two important points of interests when evaluating the effects of an intervention.By allocating patients randomly to either the IG or the CG, patient characteristics are distributed randomly between the groups, including variables potentially influencing the outcome.Hence, any difference in the outcome can be only explained by the intervention.Following this rationale, a typical observation made after randomization is the finding of equal means and standard deviations in both groups (see eAppendix 1, Figure 1).eAppendix1, Figure 1.Illustration at Baseline If the intervention is effective a mean difference is present at post-treatment assessment.
However, an important point is that not all treated patients have the same severity value after treatment.In contrast, a variance of post-treatment depression severity is observable both in the IG and the CG (see eAppendix 1, Figure 2).It is important to note that in this example, the variances are equivalent between the IG and CG.This indicates that the intervention is effective (i.e., lower observable mean depression severity) but that the extent of individual responses (i.e., variance) is equivalent in the IG and CG.Hence, variance ratios would be close to one or zero in case the logarithm is used.

eMethods 1. Model Parameters and Interpretation
Throughout the analyses we applied a three-level Bayesian meta-regression model.The key parameters of interest in the three-level regression models are 1) the intercept (μ ), which indicates whether the ratio of variance between IG and CG significantly differs from zero and, hence, substantial patient-by-treatment interactions exist, and 2) the estimate of moderators (β ) on the estimated lnVR, which indicates whether the influence of the moderator is significant.We consistently employed 95% credibility intervals (CrI) in all analyses to gauge uncertainty.The 95%-CrI of the posterior distribution of a parameter provides an interval wherein the true parameter lies with a 95% probability.If the 95%-CrI does include zero for the HTE effect size (lnVR), the assumption of different variances in IG and CG would be implausible.In addition, we accounted for between-observation variance (level II) and between-study variance (level III) of the true effect size.If 95%-CrI of level II and level III estimates include zero, the modeling of these estimates would be unnecessary, implying a simpler model structure.For an in-depth introduction to three-level meta-regression, please refer to: Assink M, Wibbelink CJM.Fitting three-level meta-analytic models in R: A step-by-step tutorial.We selected weak priors in all analyses:  ~ ℎ(0, 1)  ~ ℎ(0, 1)  ~ ℎ(0, 0.5) Note: μ (=intercept estimate), β (=predictor/moderator estimate), τ variance between outcomes within a study (level 2) and variance between studies (level 3).
To explore the impact of priors we additionally ran analyses using the posterior estimates obtained in

HTE Subgroup and Sensitivity Analysis
To investigate the role of various design and study characteristics, intervention characteristics, participant characteristics, and potential long-term HTE, we conducted subgroup and sensitivity analyses as follows: Design and study characteristics: The potential influence of the study setting (efficacy studies and effectiveness studies) was investigated in respective subgroup analyses (e.g., a subset of only effectiveness studies).As defined in previous studies [3][4][5] , we coded studies as efficacy studies if they investigated effects in highly controlled circumstances (i.e., resource-intensive setting, restrictive inand exclusion criteria, unrepresentative highly trained personnel, and standardized, strictly enforced intervention).In contrast, an effectiveness study is characterized by a real-world setting, a heterogeneous sample resulting from few to no exclusion criteria, personnel representative of usual care providers, and access to treatment as usual and other parallel interventions.In addition to the study setting, we investigated the type of control group by separately analysing RCTs following a waitlist control group, attention control, and treatment-as-usual design.Participant characteristics: For participant characteristics, we included the average age of participants in a study and the percentage of female participants as a covariate in the in the metaregression model.In addition, we investigated the baseline severity expressed in PHQ-9 (z-standardized) as a predictor of HTE.Recoding of depression instruments into PHQ-9 was performed by two independent researchers (YT and PP) using standardized conversion equations 6 and commonmetric cross-walk tables. 4,7,8ng-term HTE: Lastly, we shifted the analysis away from only post-treatment assessments to all assessments (i.e., including all assessment points of a study).Assessment time (i.e., time after baseline assessment) was included as a covariate to investigate HTE over time as a linear and quadratic predictor.Moreover, we conducted subgroup analyses in subsets only including comparisons between IG and CG within a) the first six months, b) six to twelve months, and c) over one year after the baseline assessment. 1.
Assink M, Wibbelink CJM.Fitting three-level meta-analytic models in R: A  eAppendix 7. Risk of Bias and Study Quality: Sensitivity Analysis investigate the potential impact of the risk of bias and study quality, we conducted additional sensitivity analyses on the risk of bias (i.e., overall risk of bias as a sum across all risk of bias items and item-wise analysis comparing low and high risk of bias studies) and the impact of ITT analysis (i.e., per protocol compared to intention-to-treat studies).Please see the statistical analysis section in the manuscript for the general analysis methodology.Risk of bias was analyzed analogously to the sensitivity analysis described for participant, intervention, and study design characteristics.For details on the methodology regarding the analysis of the effects of IMI, please see eMethods 4. Results on HTE and study quality are presented in eAppendix 7, Table 1.Results on the effects and study quality can be found in eAppendix 7, Table 2
From participant characteristics, only baseline severity was affecting ES (β=-0.26,95%-CrI: -0.17 to -0.36).Including the main and interactions effects of baseline severity and therapeutic guidance compared to unguided yielded a significant interaction effect between guidance and baseline severity, showing an increased impact of therapeutic guidance with increasing baseline severity
Intervention characteristics: IMI vary highly in the extent of human-provided support.Hence, we conducted subgroup analyses for RCTs including a) therapeutically guided interventions (therapeutic support by a human), b) technically guided interventions (compliance-focused support by a human), and c) unguided interventions (no support by a human).Furthermore, we investigated the theoretical background of IMI (i.e., cognitive behavioural therapy, or third-wave therapy) and the technology type (e.g., Internet-based, smartphone app based).Lastly, due to the fast developments in technology, additional subgroup analyses were conducted limited to studies published in the last decade (2013 to 2023) and five years (2018 to 2023).
Due to the large number of studies, we provide a digital version of the data set with filter options to easily sort and identify the studies and points included in the analysis respectively.Full data set, and analysis script is open-accessible at https://osf.io/u3vdn/under CC-By Attribution 4.0 international license.eAppendix 6.Individual Risk of Bias Ratings For individual risk of bias ratings per study please see the materials in the Risk of Bias folder provided on the open science framework https://osf.io/u3vdn/.Material include a) a data set including the study wise ratings, and b) a figure summarizing the ratings for each study.
In addition to the HTE analysis, we conducted a Bayesian three-level meta-regression on the effect size (ES) of IMI for depression.Therefore, we calculated Hedges' g between IG and CG (level I).

Table 1 -
. Influence of Quality on HTE Meta-Analysis

Table 2 -
Influence of Study Quality on Effects Meta-AnalysisBayesian Three-Level Meta-Regression Results for HTE in IMI for Depression.
Note.Level 1 estimates in primary outcome and subgroup analysis quantify the HTE (lnVR μ ; zero indicates equivalent variances in IG and CG).Regression results present the influence of the investigated variables on HTE (β ; zero indicates no effect on HTE).Level 2 estimates quantify how much estimates vary within studies.Level 3 estimates quantify the extent estimates vary between studies.95%-CrI quantify the 95% interval in which the true estimate lies, given the provided data.eAppendix 9. Efficacy and Effectiveness Analysis Results